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DETAILED ACTION 
Drawings 

1. The drawings were received on 08/03/04. These drawings are acceptable. 



Claim Objections 

2. Claim 10 is objected to because of the following informalities: Claim 10, line 1, "form " 
should be changed to "from ". Appropriate correction is required. 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

4. Claims 1-35 are rejected under 35 U.S.C. 103(a) as being unpatentable over Venkayala et 
al. (US Pub. No 20030212679), in view of Rosen et al. (US Patent No. 6,513,025). 

As to claims 1, 28, 35, Venkayala teaches a method of searching data in databases using 
an ensemble of models (i.e. seeds models, [0018]), said method comprising: 
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ordering models (i.e. scoring of models, [0017]) within said ensemble in order of 
prediction accuracy (i.e. Trained model 110 may also be evaluated and adjusted in order to 
improve the quality, i.e. prediction accuracy, of the model, [0019]), with the most accurate model 
being first in said order (i.e. a topmost category including a class value having a highest 
associated probability, [0010]), ([0008-0009, 0018, 0019, 0035]); 

selecting a sub-ensemble of said models that meets a given level of confidence (i.e. how 
much confidence may be placed in the prediction, [0023], the selected class values are those 
meeting the selection criteria presented in prediction parameters, [0024]), wherein said sub- 
ensemble in said order prediction accuracy ([0010, 0035]); and 

applying said sub-ensemble, in place of said ensemble, to an example to make a 
prediction (i.e. The selected class values, which are included in multi-category apply output, 
[0024]) ([0010, 0035]). 

Venkayala teaches the step of forming sub-ensembles, wherein said sub-ensembles in 
said order of prediction accuracy (i.e. model apply, [0017]). 

Venkayala does not expressly teach "models are joined together in said sub-ensemble". 
However, Rosen teaches "models are joined together in said sub-ensemble" (i.e. The most 
dependable classification model, that is the classification model associated with the 
dependability model indicating the highest dependability for the unlabeled example, is chosen to 
classify each unclassified example (col. 4, line 66 to col. 5, line 9). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Venkayala and Rosen at the time the invention was made to modify the system of Venkayala to 
include wherein models are joined together in said sub-ensemble in said order of prediction 
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accuracy as taught by Rosen. One of ordinary skill in the art would be motivated to make this 
combination in order to select the most appropriate classification model, and the prediction of 
that classification model is then accepted in view of Rosen, as doing so would give the added 
benefit of coalescing the predictions of the base models by learning the relationships between 
those predictions and the correct prediction, identifying a specific classification model as the one 
responsible for producing a final prediction and to including a simple explanation of why the 
prediction was made, and revising the selected model as the models change over time as taught 
by Rosen (col. 2, lines 31-42). 

As per claim 8, Venkayala teaches a method of searching data in databases using an 
ensemble of models (i.e. seeds models, [0018]), said method comprising: 

ordering models (i.e. scoring of models, [0017]) within said ensemble in order of 
prediction accuracy (i.e. Trained model 1 10 may also be evaluated and adjusted in order to 
improve the quality, i.e. prediction accuracy, of the model, [0019]), with the most accurate model 
being first in said order (i.e. a topmost category including a class value having a highest 
associated probability, [0010]) ([0008-0009, 0018, 0019, 0035]); 

selecting a sub-ensemble of said models that meets a given level of confidence (i.e. how 
much confidence may be placed in the prediction, [0023], the selected class values are those 
meeting the selection criteria presented in prediction parameters, [0024]), wherein said sub- 
ensemble in said order prediction accuracy, such that said sub-ensemble include only the most 
accurate models (i.e. a topmost category including a class value having a highest associated 
probability, [0010]) ([0008, 0009, 0035]); and 
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applying said sub-ensemble, in place of said ensemble, to an example to make a 
prediction (i.e. The selected class values, which are included in multi-category apply output, 
[0024]) ([0010, 0035]). 

Venkayala teaches the step of forming sub-ensembles, wherein said sub-ensembles in 
said order of prediction accuracy (i.e. model apply, [0017]). 

Venkayala does not expressly teach "models are joined together in said sub-ensemble". 

However, Rosen teaches "models are joined together in said sub-ensemble" (i.e. The 
most dependable classification model, that is the classification model associated with the 
dependability model indicating the highest dependability for the unlabeled example, is chosen to 
classify each unclassified example (col. 4, line 66 to col. 5, line 9). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Venkayala and Rosen at the time the invention was made to modify the system of Venkayala to 
include wherein models are joined together in said sub-ensemble in said order of prediction 
accuracy as taught by Rosen. One of ordinary skill in the art would be motivated to make this 
combination in order to select the most appropriate classification model, and the prediction of 
that classification model is then accepted in view of Rosen, as doing so would give the added 
benefit of coalescing the predictions of the base models by learning the relationships between 
those predictions and the correct prediction, identifying a specific classification model as the one 
responsible for producing a final prediction and to including a simple explanation of why the 
prediction was made, and revising the selected model as the models change over time as taught 
by Rosen (col. 2, lines 31-42). 
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As per claim 15, Venkayala teaches a method of searching data in databases using an 
ensemble of models (i.e. seeds models, [0018]), said method comprising: 

performing training (i.e. training/model building, [0019]) comprising: 

ordering models (i.e. scoring of models, [0017]) within said ensemble in order of 
prediction accuracy (i.e. Trained model 1 10 may also be evaluated and adjusted in order to 
improve the quality, i.e. prediction accuracy, of the model, [0019]), with the most accurate model 
being first in said order (i.e. a topmost category including a class value having a highest 
associated probability, [0010]) ([0008-0009, 0018, 0019, 0035]); 

forming sub-ensembles (i.e. model apply, [0017]), wherein said sub-ensemble in said 
order of prediction accuracy ([0010, 0035]); 

calculating confidence values of each of said sub-ensembles (i.e. to generate one or more 
scores for each row of data in scoring data. The scores for each row of data indicate how closely 
the row of data matches attributes of the model, how much confidence may be placed in the 
prediction, [0023]); and making a prediction comprising: 

selecting a sub-ensemble of said models that meets a given level of confidence (i.e. The 
selected class values are those meeting the selection criteria presented in prediction parameters, 
[0024]) ([0010, 0035]); and 

applying said sub-ensemble, in place of said ensemble, to an example to make a 
prediction (i.e. The selected class values, which are included in multi-category apply output, 
[0024]) ([0010, 0035]). 

Venkayala teaches the step of forming sub-ensembles, wherein said sub-ensembles in 
said order of prediction accuracy (i.e. model apply, [0017]). 



Application/Control Number: 10/737,123 Page 7 

Art Unit: 2167 

Venkayala does not expressly teach "joining different number of models together to form 
sub-ensembles". 

However, Rosen teaches "joining different number of models together to form sub- 
ensembles" (i.e. The most dependable classification model, that is the classification model 
associated with the dependability model indicating the highest dependability for the unlabeled 
example, is chosen to classify each unclassified example (col. 4, line 66 to col. 5, line 9). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Venkayala and Rosen at the time the invention was made to modify the system of Venkayala to 
include joining different numbers of models together to form sub-ensembles, wherein models are 
joined together in said sub-ensemble in said order of prediction accuracy as taught by Rosen. 
One of ordinary skill in the art would be motivated to make this combination in order to select 
the most appropriate classification model, and the prediction of that classification model is then 
accepted in view of Rosen, as doing so would give the added benefit of coalescing the 
predictions of the base models by learning the relationships between those predictions and the 
correct prediction, identifying a specific classification model as the one responsible for 
producing a final prediction and to including a simple explanation of why the prediction was 
made, and revising the selected model as the models change over time as taught by Rosen (col. 2, 
lines 31-42). 

As per claim 21, Venkayala teaches a service of searching data in databases using an 
ensemble of models (i.e. seeds models, [0018]), said service comprising: 
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ordering models (i.e. scoring of models, [0017]) within said ensemble in order of 
prediction accuracy (i.e. Trained model 1 10 may also be evaluated and adjusted in order to 
improve the quality, i.e. prediction accuracy, of the model, [0019]), with the most accurate model 
being first in said order (i.e. a topmost category including a class value having a highest 
associated probability, [0010]) ([0008-0009, 0018, 0019, 0035]); 

selecting a sub-ensemble of said models that meets a given level of confidence (i.e. how 
much confidence may be placed in the prediction, [0023], the selected class values are those 
meeting the selection criteria presented in prediction parameters, [0024]), wherein said sub- 
ensemble in said order prediction accuracy ([0010, 0035]); and 

applying said sub-ensemble, in place of said ensemble, to an example to make a 
prediction (i.e. The selected class values, which are included in multi-category apply output, 
[0024]) ([0010, 0035]). 

Venkayala teaches the step of forming sub-ensembles, wherein said sub-ensembles in 
said order of prediction accuracy (i.e. model apply, [0017]). 

Venkayala does not expressly teach "models are joined together in said sub-ensemble". 

However, Rosen teaches "models are joined together in said sub-ensemble" (i.e. The 
most dependable classification model, that is the classification model associated with the 
dependability model indicating the highest dependability for the unlabeled example, is chosen to 
classify each unclassified example (col. 4, line 66 to col. 5, line 9). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Venkayala and Rosen at the time the invention was made to modify the system of Venkayala to 
include wherein models are joined together in said sub-ensemble in said order of prediction 
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accuracy as taught by Rosen. One of ordinary skill in the art would be motivated to make this 
combination in order to select the most appropriate classification model, and the prediction of 
that classification model is then accepted in view of Rosen, as doing so would give the added 
benefit of coalescing the predictions of the base models by learning the relationships between 
those predictions and the correct prediction, identifying a specific classification model as the one 
responsible for producing a final prediction and to including a simple explanation of why the 
prediction was made, and revising the selected model as the models change over time as taught 
by Rosen (col. 2, lines 31-42). 

As to claims 2, 9, 16, 22, 29, Venkayala teaches said sub-ensemble includes fewer 
models than said ensemble (i.e. The selection criterion may comprise one of a topmost category 
including a class value having a highest associated probability, top N categories including N 
class values having highest associated probabilities, bottom N categories including N class 
values having lowest associated probabilities, or a set of select class values specified by the user 
and their associated probabilities and ranks, [0010]). 

As to claims 3, 10, 17, 23, 30, Venkayala teaches said confidence is a measure of how 
closely results from said sub-ensemble will match results from said ensemble (i.e. The scores for 
each row of data indicate how closely the row of data matches attributes of the model, how much 
confidence may be placed in the prediction, how likely each output, [0049]). 
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As to claims 4, 11, 18, 24, 31, Venkayala teaches the size of each sub-ensemble is 
different and has a potentially different level of confidence (i.e. The selection criterion may 
comprise one of a topmost category including a class value having a highest associated 
probability, top N categories including N class values having highest associated probabilities, 
bottom N categories including N class values having lowest associated probabilities, or a set of 
select class values specified by the user and their associated probabilities and ranks, [0010]). 

As to claims 5, 12, 19, 25, 32, Venkayala teaches the size of said ensemble is fixed (i.e. 
the selection criteria may include a limit on the number of class values that are to be selected, 
[0050]). 

As to claims 6, 13, 20, 26, 33, Venkayala teaches as the level of confidence is raised, a 
sub-ensemble that has more models will be selected in said selecting process (i.e. The selection 
criteria may be defined by desired results data, [0024]), and as the level of confidence is lowered, 
a sub-ensemble that has fewer models will be selected in said selecting process (i.e. The 
selection criterion may comprise one of a topmost category including a class value having a 
highest associated probability, top N categories including N class values having highest 
associated probabilities, bottom N categories including N class values having lowest associated 
probabilities, or a set of select class values specified by the user and their associated probabilities 
and ranks, [0010]). 
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As to claims 7, 14, 27, 34, Venkayala teaches before said selecting, calculating 
confidence values of different sub-ensembles (i.e. to generate one or more scores for each row of 
data in scoring data. The scores for each row of data indicate how closely the row of data 
matches attributes of the model, how much confidence may be placed in the prediction, how 
likely each output, [0049]). 
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Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Miranda Le whose telephone number is (571) 272-41 12. The 
examiner can normally be reached on Monday through Friday from 8:30 AM to 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham, can be reached on (571) 272-7079. The fax number to this Art 
Unit is 571-273-8300. 

Any inquiry of a general nature or relating to the status of this application should be 
directed to the Group receptionist whose telephone number is (703) 305-3900. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http ://pair-direct . uspto . gov . Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

Miranda Le 
June 12, 2006 



